Unsupervised NN approach and PCA for Background – Foreground video segmentation

MPEG-4 based video coding applications require the segmentation of each video image in its principal moving objects to be coded independently from each other. Several techniques of video objects segmentation for coding purposes have been presented in literature; all such segmentation techniques are based on the smart soft-thresholding of the motion fields, the best ones dealing with dense motion fields. Anyway, MPEG-4 based coding structures require a block based (sparse) motion field estimation. The use of block based coding structures, don’t allow fair video objects segmentation for the intrinsic inaccuracy of motion estimate the block based structure of the motion field, specially on moving object border blocks. In this context the segmentation obtained basing only on motion information is inaccurate, but it can be enhanced by the joint use of several information at hand, like color, motion, frame difference, prediction error, texture and so on. In this work a locally connected unsupervised neural network approach is presented, to obtain the segmentation of a moving video object (VO) on a fixed or slow–translating background. The purpose of background-foreground segmentation is here addressed by a split and merge like criterion: the neural network is applied to avail on correlation between several image components such as color, motion vectors and frame difference to simplify each image of the sequence into disjoint regions, thus reducing the dimensionality of the problem. The segmentation is obtained working on sub-spaces the whole subspace spanned by the several image components used to define the image, in a partition tree scheme. The second step consists on a Principal Component Analysis (PCA) to simplify the several clusters obtained in the first phase and obtain a two sets partition of the whole image. As the main information used in this phase deal principally on motion estimates and frame difference the PCA simplification step is likely to produce an image partition close to the searched BF segmentation. Preliminary results of the proposed method applied on “Foreman” video sequence seem promising. Network parameters tuning to make the proposed algorithm feasible is still needed; applications to a MPEG-4 based video coding structure is a future goal we will pursue. A complete neural network structure designed to produce the searched image segmentation, based on results of this preliminary study, is under construction.